The present invention relates generally to computer software, and particularly to methods and systems for caching of software instructions.
A computer processing unit functions by processing data elements through assorted registers in accordance with instructions provided by a computer program. The processing unit executes instructions in the form of machine language, which are typically low-level instructions used to coordinate which data elements are processed through each of the registers. Most software, however, is written in higher-level programming code, with strong abstraction from the details of the computer, which have the advantages of being human-readable and of embodying comparatively complex processing operations using relatively short commands. A compiler typically receives the high-level programming code (source code), and based upon the design of the compiler itself, generates the machine language that is readable by the processing unit.
Typically, caches are designed so that they function transparently to the software running on the processing unit. Customized hardware checks the cache for needed data, fetching data from a larger memory and managing which subset of the total data is currently in the cache. This design makes programming processing units with caches relatively easy, since the programmer is not required to explicitly manage the memory hierarchy. Hardware caches, however, have some significant drawbacks: They are complex subsystems that require substantial effort in initial design, timing closure and verification, thereby increasing time-to-market and development costs. The cache uses tags (typically containing indices of each datum in main memory that has been cached) and control logic that consume considerable area and power, which is dedicated solely to caching and is therefore unavailable for extra computation or storage, thus increasing manufacturing costs.
Use of a software-managed cache is a solution for local storage of remote data in systems that do not have a hardware cache, such as synergistic processing elements (SPEs) in the Cell broadband multiprocessor engine (developed jointly by Sony, Toshiba and IBM). The term “software-managed cache” (also referred to as a “software-based cache” or simply a “software cache”) denotes a cache that is managed in software by the processor that uses the cache, without dedicated cache management hardware. Software-managed caches can be used for caching of both data and instructions. Using such software-managed caches, a program can load the data or instructions it requires on an as-needed basis into the cache from global address space. Miller and Agarwal, for example, describe this sort of design of an instruction cache in “Software-based Instruction Caching for Embedded Processors,” ASPLOS-XII: Proceedings of the 12th International Conference on Architectural Support for Programming Languages and Operating Systems (2006).